Variational Texture Synthesis with Sparsity and Spectrum Constraints

International audienceThis paper introduces a new approach for texture synthesis. We propose a unified framework that both imposes first order statistical constraints on the use of atoms from an adaptive dictionary, as well as second order constraints on pixel values. This is achieved thanks to a variational approach, the minimization of which yields local extrema, each one being a possible texture synthesis. On the one hand, the adaptive dictionary is created using a sparse image representation rationale, and a global constraint is imposed on the maximal number of use of each atom from this dictionary. On the other hand, a constraint on second order pixel statistics is achieved through the power spectrum of images. An advantage of the proposed method is its ability to truly synthesize textures, without verbatim copy of small pieces from the exemplar. In an extensive experimental section, we show that the resulting synthesis achieves state of the art results, both for structured and small scale textures

Wasserstein Loss for Image Synthesis and Restoration

International audienceThis paper presents a novel variational approach to impose statistical constraints to the output of both image generation (to perform typically texture synthesis) and image restoration (for instance to achieve denoising and super-resolution) methods. The empirical distributions of linear or non-linear descriptors are imposed to be close to some input distributions by minimizing a Wasserstein loss, i.e. the optimal transport distance between the distributions. We advocate the use of a Wasserstein distance because it is robust when using discrete distributions without the need to resort to kernel estimators. We showcase different estimators to tackle various image processing applications. These estimators include linear wavelet-based filtering to account for simple textures, non-linear sparse coding coefficients for more complicated patterns, and the image gradient to restore sharper contents. For applications to texture synthesis, the input distributions are the empirical distributions computed from an exemplar image. For image denoising and super-resolution, the estimation process is more difficult; we propose to make use of parametric models and we show results using Generalized Gaussian Distributions

Exemplar-based Texture Synthesis: the Efros-Leung Algorithm

Exemplar-based texture synthesis aims at creating, from an input sample, new texture imagesthat are visually similar to the input, but are not plain copy of it. The Efros–Leung algorithm is one of the most celebrated approaches to this problem. It relies on a Markov assumption andgenerates new textures in a non-parametric way, directly sampling new values from the inputsample.In this paper, we provide a detailed analysis and implementation of this algorithm. The codeclosely follows the algorithm description from the original paper. It also includes a PCA-basedacceleration of the method, yielding results that are generally visually indistinguishable fromthe original results.To the best of our knowledge, this is the first publicly available implementation of thisalgorithm running in acceptable time. Even though numerous improvements have been proposedsince this seminal work, we believe it is of interest to provide an easy way to test the initialapproach from Efros and Leung. In particular, we provide the user with a graphical illustrationof the innovation capacity of the algorithm. Experimentation often shows that the path betweenverbatim copy of the exemplar and garbage growing is somewhat narrow, and that in mostfavorable cases the algorithm produces new texture images by stitching together entire regionsfrom the exemplar